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A recommender system is an algorithm aiming at giving suggestions to users 
on relevant elements or items such as products to purchase, books to read, 
jobs to apply or anything else depending on industries or situations. Recently, 
there has been a surge in interest in developing a recommender system in a 
variety of areas. One of the most widely used approaches in recommender 


systems is collaborative filtering (CF). The CF is a strategy for automatically 


creating a filter based on a user's needs by extracting desires or 
recommendation information from a large number of users. The CF approach 
uses multiple correlation steps to do this. However, the occurrence of 
uncertainty in finding the best similarity measure is unavoidable. This paper 
outlines a method for improving the configuration of a recommender system 
that is tasked with recommending an appropriate study field and supervisor to 
a group of final-year project students. The framework we suggest is built on a 
participatory design methodology that allows students' individual opinions to 
be factored into the recommender system's design. The architecture of the 
recommender scheme was also illustrated using a real-world scenario, namely 
mapping the students' field of interest to a possible supervisor for the final 
year project. 
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1. INTRODUCTION 

Recommender systems are technological instruments and strategies that can recommend things that 
a person might find useful. The recommendations may be applied to a variety of decision-making 
mechanisms, such as what to buy, what music to listen to, or what internet news to read [1]. Meanwhile, as 
described in [2], recommender systems may serve user interests in order to propose things to buy or 
investigate. There are three types of recommender systems: i) shared filtering recommender systems, ii) 
content-based recommender systems, and iii) hybrid recommender systems [3]. 

The most commonly used technique in recommender schemes is collaborative filtering [4]. The 
method of filtering or analysing objects using the views of others is known as collaborative filtering (CF). 
Calculating the similarities between users or objects is one of the key features of the CF methodology. As a 
result, CF makes suggestions to users based on the views of other users. Since the content of the goods is not 
taken into account during the recommendation process, it may also formulate nuanced recommendations. As 
a result, CF has become a common filtering tool that is used in a variety of applications [3]. 
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However, in traditional CF systems, the fundamental issue arises that concern on the amount of 
work that increases with the number of participants in the system [5]. Moreover, as discussed in the literature, 
the other issue using CF is related to the quality of the recommendation given to the user. This is because 
users need a recommendation where they can trust and refusing to use a recommender system that is not 
consistently accurate for them [3]. Despite the fact that multiple CF similarity metrics have been proposed in 
the literature, none of them have focused on individual consumer interests [6]. 

The participatory design strategy entails including consumers in the design creation process to 
ensure that the end product satisfies their needs and can be used in practice [7]. Control structures [8], 
educational [9] and medical [10] fields have also used participatory design to find solutions to complicated 
challenges, particularly when working with individuals. It lays out a framework for making the design 
process more collaborative and effective. 

Through a participatory design approach, this paper suggests a method for shaping the recommender 
system's functionality. To be more specific, the participatory design approach is used to determine the best 
similarity metric within the recommender scheme, which focuses on genuine consumer preferences. The 
demonstration and assessment of a real-world scenario was provided in order to configure the system for 
practical use. 


2. BACKGROUND 
In this section, we briefly provide background in respect to a recommender system, collaborative 
filtering, similarity measure, final year project and user studies. 


2.1. Recommender system 

According to Ricci et al. [1], the recommendation system is an application and software approach 
that helps users in selecting the best option from a large list of alternatives. This method will assist 
consumers by providing appropriate advice on different decision-making strategies. The types of things to 
buy, the books to read, and so on are examples of commonly used decision-making methods. Using a 
recommender scheme, users would be able to make an informed decision. 

A recommender system is one that gives advice and feedback to consumers when they are offered 
an alternative or a number of options to choose from when making a decision. Burke [2] describes any device 
that can provide individualised suggestions or that can assist users in identifying interesting details on items 
in a large space of possible alternatives in a customised manner as "any system that can provide 
individualised recommendations or provide the potential to support users in a personalised manner identify 
interesting information on things in a large space of possible alternatives." According to Jannach et al. [11], 
the suggested method is helpful in assisting users in matching products, as if the users were receiving support 
or advice from sales assistance, guideline, and others. Several systems make suggestions to consumers, 
according to other reports [12]-[15]. For example, a book suggestion for online shopping that suggests 
appealing websites, as well as apps that assist consumers in finding music and movies. 


2.2. Collaborative filtering 

Many commercial recommender systems have effectively deployed collaborative filtering (CF). 
These programmes suggest objects that have been preferred by other users with identical preferences to the 
actual user. As a result, various methods for developing similarity metrics, such as cosine similarity and 
correlation-based, i.e. Pearson and Spearman [16], have been proposed. Present similarity metrics use 
additional heuristic details yet seldom show global ranking behaviours on objects [16], but there is always 
room for improvement. 


2.3. Similarity measure techniques 

In the literature, some similarity metrics for recommender systems have been proposed. Sarwar et 
al. [5], for example, compared multiple item-item similarity measurements including cosine similarity and 
correlation. The user rates, which are contained in the user-item rate matrix, were used to apply these 
similarity tests. Item-based methods outperformed user-based approaches, according to the study. 

A community of researchers have obtained two different kinds of similarity measures. They are 
focused on correlation and vector cosine [17]. These approaches calculate similarity based on consumer 
reviews of the objects in question. Context correlation tests, on the other hand, do not mean whether 
consumers have common desires. As a result, a similarity metric that takes actual user preferences into 
account rather than user rates, accounts, or background information is needed [6]. 
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3. A USER STUDY APPROACH 
We briefly discuss a recommender system, collaborative filtering, similarity measures, final year 
projects, and usage experiments in this section. 


3.1. Participatory student UiTM Perlis 

This survey collected responses from 59 participants at Universiti Teknologi MARA (UiTM), Perlis 
Branch [18]. In addition, the majority of the participants are from the Bachelor of Information Technology 
(CS240) program's final year students (current and former students). We created the online survey using the 
Google Forms software because it is easy to set up and use. 

Razak et al. [19], [20] discovered that people are far more comfortable offering rankings than 
numerical values in their research. As a result, for their final year project, we also asked students to rate the 
questions in their area of interest. Students can, for example, list their areas of interest from 1 to 5, with 5 
indicating the most interested and | indicating the least interested. Students must therefore identify their 
current supervisor (SV) and co-supervisor. For instance, User | has ranked the area of interest as 2 for 
Network design (N1). For simplicity, we use the short version of the word abbreviation for the area of interest 
(Nl=Network design, N2=Network — security, N3=Network administration, _N4=Microcontroller, 
M1=Multimedia, M2=Multimedia Courseware, M3=User Interface design, M4=Augmented Reality, 
W1=Web Information System, W2=Decision Support System, W3=E-Learning, W4=E-Commerce, 
Al=Expert System, A2=Neural Network, A3=Machine Learning, A4=Natural Language Processing, 
MO1=Mobile Application, MO2=Mobile Learning, MO3=Mobile Games, MO4=Internet of Things). It 
should be noted that all students already have the supervisor, and indeed some of them already completed this 
FYP subjects. The sample ranking of students (U 1-U59) for the area of interest can be seen in Table 1. 


Table 1. The sample participants ranking on the area of interest for their FYP 
Area of Interests 


Users 1 ND N3 N4 MI M2 M3 M4 WI W2 W3 W4 Al A2 A3 A4 MOL MO2 MO3 Mo4 SY 
Ui 0.96 ° 3.3%, 3° 3 =a > Ot; 390-43. B23. Ot oe 3 3 3 3. S2 
Ud* O° 2 9 eB BB A ee 4 2 3 $2 
U3 2 1 1 1 4 4 4 3 5 4 5 5 3 12 3 «5 5 4 3. $6 
W432 638 S30 BRB, 6B Be 8B. SS) Bl BL Be Ng. Bye a, 3 3 3 3. S-ll 
US: <3. 39 3. BO Aa ae > Be SS 3 gs Ag) eg Cae or Age a 2 2 5 §-14 
USS 3 32 3-30 3) 3 Be OH Be Be BB Bh ge Be - B 3 3 3 $6 
U56 2 4 3 4 5 3 4 4 4 4 5 3 3 2 4 2 «4 4 4 5 §-20 
U57 3 4 4 2 4 4 4 2 4 3 3 2 2 3 2 2 «3 3 2 5 §-20 
USS 2 8 83 4B Be Bs OBE. A Oh BY BS BS 4 5 5 S-18 
U59 4 4 4 4 5 5 4 4 4 3 4 3 3 3 3 3 «4 4 4 5 $6 


3.2. Creating a supervisor rating-participatory design approach 

This section explains how to obtain supervisory status over all of the areas of concern raised in the 
final year programme. In this case, we use a participatory design approach to help with the supervisor 
ranking production. Table | shows that only 20 supervisors were selected for their final year project out of 59 
users. Several students choose the same supervisor but ranked him or her differently for their field of 
concern. This is due to the fact that the same supervisor might have several research interests and can 
supervise the potential students in several research area. 

In order to generate the standard supervisor ranking, we use a simple aggregation function to 
combine the various rankings provided from students for the same supervisor. For example, Users 1 and 2 
chose S-2 as their supervisor with different ranking for the area of interest. Therefore, the aggregation mean 
was adopted to combine both rankings for the area of interest for S-2. The sample supervisor ranking of the 
area of interest is presented in Table 2. 


Table 2. The sample supervisor ranking produced from participatory design approach 
Area of Interests 
NI N2 N3 N4 Mi M2 M3 M4 WI W2 W3 W4 Al A2 A3 A4 MOl MO2 MO3 MO4 SV 
4 3 4 4 35 35 35 3 4 35 4 4 4 3 4 4 3.5, 3.5, 4 4 S-1 
2 
2 


3. 2") 436 «BS. 35. 3 3 i 3, 3° “Bids 336.3) T3n- 18 es) 3:3 2.5 3 S-2 


3 3 2) AT 435 43° -43 33". 3 4 3.7 2 


Pree 
ww 
w 
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bo 


43 43 37 $3 
35 35 3 #3 S17 
35 35 35 3 $-18 


3 3 3 3 S-19 
3.5 3.3. 34 44 S-20 


Wo WW Wes 
WR WW: 
Wo WW Wes 
WW WwW Ws: 
WwW WWW: 


36 34 35 34 #34 #31 3 2.5 
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4. PRELIMINARY RECOMMENDER SYSTEM-EUCLIDEAN DISTANCE 

This section addresses a tentative recommender framework that can be used to pair final-year 
project students with others who share their interests. The potential supervisors were presented in previous 
work [21]. This is a simple way to measure the similarity between students’ interest with the potential 
supervisor. As for the initial approach, the recommender system is composed based on ad hoc similarity 
measure that is a Euclidean distance. 


4.1. Euclidean distance score 

The interval between two points that denote vector values (x,,y,) and (x2, yz) is known as the 
Euclidean distance [22]. Euclidean distance is a useful algorithm since it is similar to object distances 
determined in the real world [23]. The formula for Euclidean distance is (1), where p is the number of data 
points, d is the distance function, and s, is the data points’ part value. 


d(p,, P2) = Dieitem(Sp, = Sp, )* (1) 


Although Euclidean distance is most commonly expressed in two-dimensional or three-dimensional 
space, it can also be used to calculate distance between multi-dimensional variables. With a basic theorem, 
the Euclidean distance between points can also be converted to quantify similarity: 


1 


Fa SRE (2) 
1+d(p1,p2) 

The conversion to similarity score would yield a result between 0.0 and 1.0, with a value near 1.0 
representing total similarity and a value near 0.0 representing complete dissimilarity. This is in reference to 
the measurement distance of distance, where 0.0 indicates a comparable object and a higher number 
(unbounded) indicates a different item [24]. 


5. A GENERIC RECOMMENDER SYSTEM-BEYOND EUCLIDEAN DISTANCE 
A key aspect of similarity measure is the need to assess in order to presented for a generic 
recommender system. In this case, the similarity measure can be computed as shown in (3): 


Similarity measure = S(A, B) (3) 


where, S is represents similarity measure strategy, A is a set of a vector number and B is another set of a 
vector number. Although a similarity measure based on Euclidean distance can be used as a default technique 
(see section 4.1), we suggest a participatory architecture approach to extract the appropriate similarity 
measure within the recommender system in order to achieve a generalised recommender system as viewed by 
individual users. Several similarity measures of collaborative filtering may be used within the recommender 
system. In this paper, the alternative similarity measure, S as in (3) will represent necessary similarity 
measure, namely Pearson correlation, Spearman correlation and Cosine similarity described in the following 
subsection. 


5.1. Pearson correlation 

The first similarity measure that explores in this paper is the Pearson correlation. The Pearson 
correlation (rdetermines the strength of a relationship between two variables, with +1 enoting a strong 
positive relationship (more similar) and —1 denoting a strong negative relationship (more dissimilar) [25]. It 
can be computed as shown in (4): 


Yer i-*)(Vi-) 


r= (4) 
[ehs@i-B? [EE 01-7? 


The (4) measures the similarity between two variables x and y. 
5.2. Spearman correlation 


The next similarity measure that will be explored in this paper is the Spearman correlation. 
Likewise, the Spearman correlation (rs) computed the similarity between two set of variables. The rs 
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statistic will be +1 if the two variables are identical (complete similar) and —1 if the variables are opposite 
(complete dissimilar) [25]. It is computed as shown in (5): 


= — 6k einyo* 

rs(%y) =u rk(rk2-1) (3) 
Likewise, the (5) may be used to measures the similarity between two variables x and y. Note that rk is the 
number of rankings in variables. 


5.3. Cosine similarity 
The cosine similarity measure has also been explored in this paper. The cosine similarity to compare 
the two vectors, can be defined as shown in (6): 


cos(x,y) = —— 224 6) 
Vey (xp)? |L1071)? 


Moreover, the (6) may be used to measures the similarity between two variables x and y. It should be noted 
that the Euclidean distance described in section 4.1 will be used in conjunction with the similarity measure 
technique for the remainder of this paper in optimizing the recommender system features. 


6. EXPERIMENT AND RESULTS-PARTICIPATORY DESIGN APPROACH 

To derive the criteria for the recommender system, i.e., similarity measures, we suggest a 
participatory architecture approach. As previously said, participatory design is a method of including 
consumers in the design creation process in order to ensure that the end product meets their requirements [7], 
[20]. A participatory design process is described in this section, and it consists of two main steps: i) 
measuring the recommender system using multiple similarity measures; and ii) matching the recommender 
system (similarity measures) with the participatory users. 

In this experiment, we use the same example as presented earlier in section 3 that is to recommend 
the potential supervisor based on a student's area of interest. We have used the participations ranking data on 
the area of interest and the supervisor rankings, as shown in Tables | and 2, respectively. The aim was to 
investigate the alternative to the similarity measure within the recommendation system. However, for data 
participation ranking, we have eliminated four rows of data, namely Users 4, 22, 24 and 55 as they give equal 
rank for all area of interest. This is to avoid unwanted errors in the computation of the alternatives to the 
similarity measure. This experiment is now presented in the following subsections studies. 


6.1. Measuring the recommender systems using several similarity measures 

This section was conducted to explore various similarity measures within the recommender system, 
as described in section 5. Four different similarity measures, namely Euclidean distance, Pearson, Spearman 
and Cosine, were explored. Notably, these similarity measures were used in a recommender system to 
suggest the potential supervisor based on the student area of interest. 

Precisely, the similarity measure was used to calculate the similarity between the area of interest, as 
in Tables 1 and 2, respectively. For instance, the area of interest for User | as in Table 1 is computed with the 
area of interest for 20 supervisors as in Table 2, aiming to suggest a suitable potential supervisor for students 
based on several similarity measures. Table 3 shows the corresponding cosine similarity measure. In this 
case, the result suggests S-5 as a supervisor for User 1 as it showed the highest cosine similarity measure. In 
general, Table 4 shows the summary of the result of the recommender systems by using several similarity 
measure strategies. 


Table 3. The computed cosine similarity measure for U-1 


Computed cosine Supervisor 
0.9913 S-1 
0.9904 S-2 
0.9805 S-3 
0.9787 S-4 
0.9947 S-5 
0.9867 S-6 
0.9725 S-19 
0.9931 S-20 
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Table 4. The sample result of recommender system using several similarity measures 
Similarity measure strategy 


User 


Euclidean Distance Pearson Spearman Cosine 
Ul S-2 S-2 S-2 S-5 
U2 S-2 S-2 S-2 S-2 
U58 S-8 S-8 S-8 S-8 
U59 S-6 S-6 S-6 S-6 


6.2. Matching the recommender system (similarity measures) with the participatory users 

This step was performed to assess the degree of consensus between the output from the participatory 
user study in section 3.1 and the recommender system in section 6.1 using multiple correlation measure 
strategies. In particular, we calculated the agreement scores between Tables | and 4 outcomes. For instance, 
User 1 prefer S-2 for his/her supervisor, as in Table 1, while the similarity measure of Pearson as in Table 4 
recommends S-2 for the supervisor. Since both results are identical, consequently the agreement score for 
them is 1. However, if the result is not similar, the score will be 0. The complete results of this agreement 
score for both qualities can be seen in Table 5. The agreements are summarised in the last two rows, with the 
mean and standard deviation (SD) for each column. From Table 5, it can be seen that the Pearson similarity 
measure achieves the highest average agreement scores with the value of 0.6364. That is, the majority of user 
responses are more similar to the ratings received using the Pearson similarity metric. 


Table 5. The sample agreement scores between the preferences given by each of the users 


and the preference indicated by several similarity measures 
Similarity measure strategy 


Use Euclidean Distance Pearson Spearman Cosine 
Ul 1 1 1 0 
U2 1 1 1 1 
U3 0 0 0 0 
U58 1 1 1 1 
U59 1 1 1 1 
Mean 0.4364 0.6364 0.6000 0.5818 
SD 0.5005 0.4855 0.4944 0.4978 


7. DISCUSSION 

Through the use of a participatory design process, we investigated a new approach to shaping 
recommender system features (similarity measure strategy). The key goals of the experiments are to 
i) explore the similarity measure approach and ii) optimise the similarity measure within the recommender 
method. It should be remembered that the supervisor rating data in this report were derived from the results 
of the user study presented in section 3.2. 

For the first step, several similarity measures within the recommender system were adopted namely 
Euclidean distance, Pearson, Spearman and Cosine to help in finding the potential supervisor for the final 
year students based on their area of interest. The result showed that these similarity measures produced the 
diversity answer in recommending the supervisor for 54 students, as presented in Table 4. As can be 
observed from Table 4, the result produces 35% of the similar trend, which can be seen at Users 2, 6, 10, 12, 
14, 17, 20, 21, 25, 26, 27, 28, 33, 37, 44, 46, 47, 48, 52, 58, and 59. However, at this stage, it is not clear 
which similarity measure is the best to represent the student perception on the selection of supervisor based 
on their area of interest. 

For the second step, we used an agreement score to reveal the similarity measure strategy that is best 
to be used with the recommender system. That is also to make sure that the strategy is matched with students’ 
perception in selecting the supervisor. As a result, Table 5 displays the agreement score between each user's 
interests and the preference suggested by multiple similarity indicators. Based on this result, we found that 
the Pearson similarity measure produced the highest mean score than others with the value of 0.6364. This 
also could indicate that the Pearson similarity measure is the best matched with the user preference to be 
employed within the recommender system, particularly in this example. Although the proposed framework 
promises to refine the recommender system, it is obvious that further work is needed for improvement in the 
future in order to produce a comprehensive recommender system. 
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8. CONCLUSION 

In conclusion, the structure for shaping the recommender method using a participatory design 
approach is a major contribution of this study. Correctly, the technique is used to optimise the similarity 
measurements strategy within the recommender method, including Euclidean distance, Pearson, Spearman, 
and Cosine similarity. The suggested method was tested using a case analysis of the recommender method in 
comparing students' interests with prospective supervisors. We consider using the Pearson similarity measure 
technique inside the recommender framework based on the existing proof. We want to do further research 
involving various case studies of more complicated and other similarity measurements in the future. We 
expect to learn more about various implementations of the generic architecture and, as a result, achieve a 
better understanding of the recommender system. 
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